Exploratoring environmental data
RJ
ID
path_to_ncdf## [1] "./DataBaseOllieNew/ID101/Trip_BC1/netCDF/"Pressure
Some context for each track's pressure data
Source: Ollie's notebooks and email exchanges between Jelle, Ollie and me
- The data are not calibrated yet.
- Since there should be more or less 5-min segments of data every hour, the raw data at 10 Hz are first divided into 5 minute segments (or 15, but it will only contain 5)
- Data cleaning is done on these segments. So, each segments:
- Assuming that barometric pressure distributes normally --Jelle said it's a safe assumption --, outliers are filtered out by first obtaining z-scores (gives you an idea of how far from the mean a data point is. But more technically it's a measure of how many standard deviations below or above the population mean a raw score is) and keeping the observations whose absolute z-scores are below 3.
- Then, if consecutive observations show a difference greater than 2hPa (meaning a ~20m difference), the following value is replaced by the preceding value.
- Resampling is done at 1 Hz. It is basically done by fitting sinusoids to the data (Fast Fourier Transform) and then taking value points. We're using the obspy.resample function in python which is a wrapper of scipy.signal.resample
Everything is saved into a netCDF file
Time series
## median_BaromP mean_BaromP min_BaromP max_BaromP first_Qu_BaromP
## 1 1015.609 1016.05 1010.259 1020.154 1013.959
## third_Qu_BaromP mean_IBI_sec mean_samples_within_bout tot_group_n
## 1 1018.885 3402.867 3004.444 18Variability within each 5 mins bout in BP
black point: mean blueish bars for min and max greenish bars for q1 and q3 greenish dash lines link medians
Now without max and min
black points: mean squares medians greenish bars for q1 and q3 greenish dash lines link medians
Data to check
Here is plotted how data should look like in first plot
Second plot shows the data within each bout for which the median and the mean differs by more than 0.3hPa.
